Simultaneous Recovery of Shape, Motion and Grouping by Applying Rank Constraints

Suppose there are two objects which are moving independently and rigidly in 3D space. Assume we have an orthographic camera. The image plane is perpendicular to Z axis and passes from the origin. What we are given as input is the projection of 3D object points to the image plane at each time step, as a sequence of images. So basically we are given a video of two objects moving, and our goal is to recover the following information: group points as seperate objects, estimate the object shapes, and recover their motion in R space. We describe each in more detail below: Grouping Points: Every pixel location in each image corresponds to projection of a 3D point of one of the two objects to the image plane. We want to group the set of points in all images that correspond to each object. Object Shapes: Since objects are rigid, we de ne a local coordinate frame in R for each object. Our goal is to project back the pixel locations corresponding to each object, to their corresponding 3D location in its local coordinate frame. Motion: We like to recover the rotation and translations applied to each object at each frame.